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Abstract 

Firms  that  design  mechanical  and  electromechanical  products  confront  a  variety  of  difficult  issues  in 
their  prototyping  activities.  For  a  given  part,  how  can  a  choice  among  fabrication  technologies  be 
made?  Where  should  investments  in  new  prototyping  technology  be  focused?  How  can  new  and  existing 
prototyping  technologies  he  evaluated?    Our  primary  goal  has  been  to  develop  a  systematic  method  of 
evaluating  prototyping  processes  in  order  to  determine  the  best  process  for  a  given  situation.  A 
secondary  goal  has  been  to  map  the  "space"  of  prototyping  processes  in  order  to  determine  future  process 
development  needs.  Using  data  from  a  field  study  at  the  Kodak  Apparatus  Division,  we  have 
developed  a  systematic  method  for  evaluating  and  selecting  prototyping  processes.  Our  data  is  drawn 
from  (Da  user  survey  of  prototyping  perceptions  and  needs,  (2)  a  survey  to  determine  the  importance  of 
various  prototype  part  performance  attributes,  and  (3)  estimates  of  the  fabrication  time,  cost,  and  part 
performance  for  104  parts  and  four  prototyping  processes. 
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1.  INTRODUCTION 

Prototyping  is  one  of  the  most  critical  activities  in  new  product  development.   Firms 
that  design  mechanical  and  electro-mechanical  products  coiifront  a  vciriety  of 
difficult  issues  in  their  prototyping  activities.  Some  of  these  issues  are  tactical:   For  a 
given  part,  how  can  a  choice  among  fabrication  technologies  be  made?  Some  of  the 
issues  are  strategic:   Where  should  investments  in  new  prototyping  technology  be 
focused?  How  can  new  and  existing  prototyping  technologies  be  evaluated?  We 
believe  that  there  are  useful  conceptual  frameworks  and  fundamental  relationships 
that  can  be  developed  and  used  to  support  prototyping  decisions,  even  though  broad 
generalizations  are  likely  to  fail  in  some  particular  situations.   Such  a  framework  is 
particularly  important  today  as  lead  times  in  product  development  become  more 
critical  [Fitzgerald87,  Ulrich91]  and  as  novel  prototyping  technologies  emerge 
[Dickerson88,  Kutay90,  Lindsay90]. 

This  paper  describes  research  addressing  these  issues.  Our  primary  goal  has  been  to 
develop  a  systematic  method  of  evaluating  prototyping  processes  in  order  to 
determine  the  best  process  for  a  given  situation.  A  secondary  goal  has  been  to  map 
the  "space"  of  prototyping  processes  in  order  to  determine  future  process 
development  needs. 

Our  basic  methodology  has  been  to  study  prototyping  in  the  context  of  an  industrial 
manufacturer.  By  using  a  data-directed  approach  based  on  existing  parts  and 
processes,  we  have  been  able  to  provide  empirical  support  for  a  systematic  method 
of  evaluating  and  selecting  processes.  Our  data  is  drawn  from  (1)  a  user  survey  of 
prototyping  perceptions  and  needs,  (2)  a  survey  to  determine  the  importance  of 
various  prototype  part  performance  attributes,  and  (3)  estimates  of  the  fabrication 
time,  cost,  and  part  performance  for  over  104  parts  and  four  prototyping  processes. 
We  focus  on  parts  intended  to  be  injection  molded;  the  prototyping  processes  on 
which  we  concentrate  are  computer-aided  design  solid  modeling  (CAD), 
stereolithography  (STL),  computer  numerical  controlled  machining  (CNC),  and 
rubber  molding  (RM). 

The  paper  has  four  sections.   In  the  balance  of  the  introduction  we  define 
prototyping,  present  a  conceptual  framework  for  evaluating  prototypes  and 
prototyping  processes,  and  describe  our  research  focus.  In  the  second  section  we 
present  our  approach  and  methodology.   The  third  section  details  our  results.   The 
final  section  lists  our  conclusions  and  suggests  areas  for  future  research. 

What  is  Prototyping? 

Technically,  a  prototype  is  the  first  thing  of  its  kind.  But  "prototype"  has  come  to 
mean  many  different  things  in  the  context  of  product  development.   For  the 
developer  of  commercial  satellites,  the  prototype  may  be  the  final  product.   At  the 
other  extreme,  the  development  of  a  new  ballpoint  pen  may  involve  more  than  10 
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prototypes  before  the  desigr.  is  finalized.  Some  prototypes  are  used  to  assess  styling, 
some  are  used  to  perform  life  testing,  some  are  used  to  verify  fit.  Other  words 
frequently  used  to  describe  the  function  of  a  prototype  include  breadboard,  model, 
and  mock  up.  Each  is  an  example  of  a  prototype;  each  emphasizes  a  different  aspect 
of  what  a  prototype  does.  In  our  definition  of  a  prototype  we  include  both  electroruc 
and  physical  representations  of  the  part  or  product. 

In  most  cases  prototypes  are  built  to  answer  questions.  In  the  engineering  context,  a 
prototype  proves  a  technology  or  its  application.   In  the  marketing  context,  the 
prototype  is  a  vehicle  through  which  the  marketer  simulates  customer  response  or 
finalizes  design  requirements.   In  the  manufacturing  context,  a  prototyp>e  proves  a 
process  or  procedure.   Effective  prototyping  requires  understanding  which  questions 
to  ask,  when  to  ask  them,  and  how  to  answer  them. 

Different  types  of  prototypes  are  used  in  many  different  ways  to  address  different 
types  of  questions.  Electronic  prototypes  are  often  used  for  simulation.  They  may  be 
subjected  to  finite  element  analysis,  mass  properties  calculations,  tolerance  analysis, 
assembly  analysis,  or  motion  analysis.   Some  prototypes  are  used  for  verification; 
often  analysis  and  simulation  have  been  used  to  make  many  design  decisions,  and 
the  prototype  is  used  to  detect  unanticipated  problems.  Some  prototypes  are  used  to 
perform  tests;  they  may  be  used  for  functional  testing,  customer  perception  testing, 
life  testing,  assembly  planning,  etc.  Other  prototypes  serve  as  crystal  balls  to 
anticipate  future  problems;  they  may  be  used  to  prepare  for  tooling  design,  or  to 
compare  the  evolving  product  with  customer  needs. 

Framework 

Product  development  professionals  have  highly  varied  prototyping  needs. 
Designers  concerned  with  styling  have  needs  different  from  those  of  mechanism 
designers.   Prototypes  to  support  conceptual  design  have  requirements  different 
from  those  used  in  pre-production.   The  design  of  some  parts  may  demand 
prototypes  with  very  tight  tolerances,  while  other  parts  may  require  special  material 
properties.   The  required  quantity  is  important  in  evaluating  processes;  some 
situations  may  require  only  one  part,  some  may  require  100  parts.   Finally,  in  every 
case  the  user  of  a  prototype  cares  to  some  extent  about  the  cost  and  time  required  to 
procure  a  prototype. 

To  capture  some  of  these  issues,  we  evaluate  prototyping  processes  along  three 
dimensions:    part  performance,  unit  cost,  and  lead  time.    We  view  part  performance 
as  an  aggregate  measure  of  the  fidelity  of  the  properties  of  a  prototype  part,  made 
with  a  particular  process,  with  respect  to  the  properties  the  part  would  have  if  it 
were  made  with  the  intended  final  production  process.   This  performance  measure 
is  composed  of  attributes  such  as  appearance,  material  properties,  and  dimensional 
accuracy.  Using  an  importance  weighting  for  each  attribute,  derived  from  a 
particular  user's  needs,  the  performance  of  a  process  with  respect  to  each  attribute 
can  be  combined  into  a  single  scalar  part  performance.  Part  cost  and  lead  time  could 
also  be  considered  to  be  dimensions  of  performance,  but  since  designers  often  think 
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of  them  independently  of  physical  part  attributes,  we  chose  to  separate  them  from 
other  performance  measures. 

More  formally,  associated  with  each  part,  i,  fabricated  with  a  particular  process,  there 
is  an  associated  cost,  c,,  lead  time,  f,,  and  performance  vector  A,-.   The  performance 
vector  for  the  part  and  process  is  composed  of  the  performance  of  the  process  with 
respect  to  each  of  the  attributes,  ay ,  or 

ai 
an 


A,= 


The  set  of  attributes  used  to  define  the  vector  may  vary  depending  upon  the 
application  or  organizational  context.   Typical  attributes  might  be  strength,  stiffness, 
density,  color,  surface  finish,  etc.  Each  Uj  is  assigned  a  value  from  0  to  1  representing 
the  degree  of  fidelity  of  the  prototyping  process  to  the  final  production  process  for 
the  particular  part.  A  score  of  0  is  a  poor  match  and  1  is  a  iperieci  match. 

For  a  given  part,  the  attribute  importance  weighting  vector,  W,  contains  the 
importance  weightings  for  each  of  the  attributes,  or 

W„ 

Each  Wj  is  assigned  a  value  from  0  to  1  where  0  is  unimportant  and  1  is  very 
important.  For  example,  for  a  part  to  be  used  in  a  structural  test,  stiffness,  strength, 
and  dimensional  accuracy  may  be  very  important  while  non-mechanical  material 
properties  like  electrical  conductivity  may  be  unimportant. 

The  dot  product  of  the  performance  vector  and  the  transpose  of  the  importance 
vector  yields  the  scalar  part  performance,  pi . 

A,-  •  WT  =  Pi 

The  determination  of  the  appropriate  weights  in  the  importance  vector  is  thus 
crucial  to  the  use  of  this  method.   Each  department  in  an  organization  could 
conceivably  use  a  different  weighting  vector  to  determine  jjerformance  of  a  process 
for  its  applications.   A  weighting  vector  might  even  be  different  for  different  parts 
within  the  same  department  (e.g.  a  duct  versus  a  bracket).   The  weighting  vector 
may  vary  with  time  as  well;  the  vector  used  at  the  concept  stage  of  development 
would  differ  from  that  used  during  the  pre-production  phase.      For  example,  a 
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"breadboard"  prototype  intended  only  to  verify  a  kinematic  function  might  have  an 
associated  weighting  vector  that  emphasizes  dimensional  accuracy,  while  a  pre- 
production  prototyp>e  intended  to  test  consumer  response  might  have  an  associated 
weighting  vector  that  emphasizes  surface  finish. 

In  our  study,  we  determined  the  relevant  performance  attributes  and  the 
importance  weightings  for  a  particular  class  of  parts  in  a  particular  company.   Then, 
based  on  estimates  of  the  time,  cost,  and  p>erformance  of  four  processes  on  over  one 
hundred  parts,  we  were  able  to  determine  the  average  performances,  costs,  and 
times  for  the  processes.  This  information  allows  us  to  prescribe  a  prototyping 
decision  procedure  for  this  particular  set  of  attributes  and  importance  weightings. 

Focus 

We  focus  on  the  prototyping  of  plastic  piece  parts.    All  of  our  data  were  gathered  at 
the  Apparatus  Division  of  the  Eastman  Kodak  Company  (KAD)  in  Rochester,  New 
York.   Although  product  design  involves  a  diverse  set  of  prototyping  applications, 
processes,  and  needs,  we  focus  on  a  particular  class  of  prototyping  problems  in  a 
specific  industrial  setting  in  order  to  generate  meaningful  results.     While  the 
specific  results  are  valid  only  for  this  setting,  the  methodology  can  be  applied  to 
other  types  of  parts  and  processes.    Evaluating  prototyping  technologies  for  printed 
circuit  boards,  metal  shafts,  or  food  products  would  require  performance  vectors 
substantially  different  from  those  of  plastic  parts,  but  we  believe  the  basic  framework 
would  remain  valid  and  useful. 

The  Parts 

All  of  the  parts  used  in  our  study  were  intended  to  be  plastic  in  final  production. 
For  over  85%  of  the  parts,  the  final  production  process  had  been  determined  to  be 
injection  molding.  For  the  balance  of  the  parts,  the  final  process  had  not  been 
determined,  or,  in  a  few  cases,  other  processes  such  as  vacuum  forming  were  to  be 
used.  The  parts  were  obtained  from  a  database  of  parts  at  the  stereolithography 
laboratory  at  KAD,  and  were  drawn  from  several  types  of  products  including 
photocopiers,  medical  imaging  equipment,  and  printers.   The  parts  varied  in 
complexity,  size,  and  shape.   We  believe  that  the  part  sample  represents  a  uniform 
distribution  of  part  types  for  injection  molded  parts  in  medium  volume  products. 
Figure  1  shows  two  typical  parts  from  the  sample. 

The  Processes 

The  study  was  based  on  four  prototyping  processes:  computer-aided  design  solid 
modeling  (CAD),  stereolithography  (STL),  computer  numerically  controlled 
machining  (CNC),  and  room  temperature  vulcanizing  rubber  molding  (RM).   It 
should  be  noted  that  the  processes  are  not  entirely  independent:   STL  and  CNC 
depend  on  a  CAD  model,  and  RM  requires  a  master  (often  generated  by  STL  or 
CNC).  The  processes  are  described  briefly  below. 
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Figure  1:  Sample  parts. 
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CAD  solid  modeling.   For  our  study,  we  assume  that  a  part  design  exists;  we  base  the 
prototype  cost  and  time  performance  for  CAD  on  the  resources  required  to  translate 
a  defined  design  into  a  solid  model.   In  many  firms,  solid  modeling  is  becoming  the 
standard  means  of  creating,  representing,  and  storing  design  data  [Hurt89,  Stix91];  in 
these  settings,  there  would  be  no  additional  cost  or  time  required  to  create  a  CAD 
solid  model  for  prototyping  purposes.  As  a  baseline  case,  we  include  only  the 
visualization  and  geometric  reasoning  capabilities  of  solid  modeling.    With  enough 
time  and  money,  simulation  and  analysis  packages  can  be  used  to  understand 
almost  every  performance  aspect  of  a  part  without  building  a  physical  prototype.  At 
KAD  these  tools  are  typically  used  on  only  the  most  critical  parts  in  a  product.  For 
this  reason,  and  because  there  is  so  much  variability  in  both  the  available  software 
and  in  user  skill,  we  confine  our  definition  of  CAD  as  a  prototyping  process  to  the 
basic  functionality  of  a  solid  modeler.  In  the  final  section  of  the  paper  we  discuss  a 
set  of  dear  opportunities  for  CAD  to  supplant  physical  prototyping. 

Stereolithography.   Stereolithography  is  a  process  by  which  solid  plastic  parts  are 
created  by  building  a  stack  of  cross  sections.  Each  cross  section  is  fabricated  by  using  a 
laser  to  cure  a  thin  layer  of  a  liquid  ultraviolet-sensitive  photo-polymer.   Estimates 
of  STL  parts  were  made  based  on  the  performance  of  a  3D  Systems,  Inc.  SLA500 
machine.   There  are  currently  only  a  few  polymers  that  can  be  used  in  the 
stereolithography  process  [Lindsay  90].  Fabricating  an  STL  part  requires  a  CAD 
model  of  the  part. 

Computer  Numerically  Controlled  Machining.    Evaluation  of  CNC  parts  assumed 
fabrication  on  a  lathe  or  five-axis  milling  machine.   Although  many  parts  could  be 
built  on  a  three-  or  four-axis  machine,  five  axes  were  assumed  in  order  to  minimize 
the  number  of  fixtures  and  setups.  Some  of  the  parts  required  setup  on  both  a  lathe 
and  a  mill  [Ford90]. 

Rubber  Molding.  Many  plastic  parts  can  be  made  by  casting  a  polymer  such  as 
polyurethane  in  a  silicone  rubber  mold.   This  process  involves  first  casting  the 
rubber  mold  around  a  part  master  and  then  filling  the  resulting  mold  cavity  with 
the  desired  part  material.   Molding  compounds  are  available  that  approximate  the 
properties  of  many  thermoplastics  used  in  injection  molding.   At  KAD,  both  CNC 
and  STI.  are  used  to  create  the  part  master. 

2.  APPROACH  AND  METHODOLOGY 

We  used  estimates  and  surveys  to  answer  the  following  questions  for  our  focused 
domain: 

•  What  attributes  are  important  for  prototype  parts  (i.e.  What  attributes 
make  up  A)? 
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•  What  are  the  relative  importances  of  these  properties  (i.e.  What  is 
W)? 

•  How  well  do  the  different  processes  perform  with  respect  to  these 
importances  (i.e.  What  are  the  values  for  the  ay 's  in  A?)? 

•  What  are  the  lead  times  and  costs  for  the  processes  (i.e.  what  are  c  and 
0? 

A  User  Survey  of  Prototyping  Perceptions  and  Needs  was  used  to  identify  critical 
attributes.   An  Attribute  Importance  Survey  was  developed  to  establish  the 
importance  of  each  attribute  at  various  phases  of  development  (the  values  of  the 
elements  of  W).  A  Process/Part  Evaluation  was  designed  to  determine  cost,  lead 
time,  and  process  performance  for  each  attribute  (the  values  for  c,  t,  and  the 
elements  of  A). 

User  Survey  of  Prototyping  Perceptions  and  Needs 

In  order  to  gather  general  information  about  prototyping  perceptions  and  needs,  we 
conducted  an  informal  written  survey  of  24  designers,  engineers,  and  managers  at  6 
of  the  lines  of  business  within  Eastman  Kodak.   The  survey  asked  for  such  things  as 
a  definition  of  "prototype",  the  purpose  of  prototypes  within  the  organization, 
material  and  process  needs,  standard  procedures  for  procuring  prototypes,  and  the 
characteristics  of  the  ideal  prototyping  facility. 

Based  on  the  responses  to  this  survey  we  identified  the  performance  attributes  given 
in  the  right  column  of  Table  1.   In  order  to  simplify  our  presentation  of  the  results, 
we  have  aggregated  the  attributes  into  four  higher-level  categories. 

Table  1:  Performance  Attributes  Identified  in  the  User  Survey 


aggregate  attributes 

constitutive  attributes 

appearance 

overall  part  appearance 

strength  /  stiffness 

strength 
stiffness 

form  accuracy 

dimensional  accuracy 
geometric  accuracy 
surface  firush 

feature  definition 

material  properties 
(other  than  strength  and  stiffness) 

hardness 

v^eight 

melting  temperature 

opacity 

conductivity 

color 
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Attribute  Importance  Survey 

A  written  survey  was  sent  to  over  900  designers,  engineers,  tool-  and  instrument- 
makers,  and  project  managers  at  Eastman  Kodak.   171  (18%)  of  the  surveys  were 
returned.   The  survey  asked  each  respondent  to  rate  the  importance  of  each  of  the 
attributes  in  Table  1  at  the  concept,  engineering,  and  pre-production  phases  of 
development.    At  each  phase,  each  attribute  received  a  ranking  from  "unimportant" 
to  "very  important."   The  survey  also  requested  information  about  the  respondents' 
primary  function  (design,  engineering,  or  management)  and  discipline  (mechanical, 
electrical,  optical,  etc.).  Of  the  171  respondents,  143  identified  themselves  as 
involved  in  a  "mechanical"  discipline,  12  specified  "electrical,"  7  specified  "optical," 
and  1 1  specified  "other."   Of  the  143  in  the  mechaiucal  discipline,  52  specified  that 
"design"  was  their  primary  fimction,  63  specified  "engineering,"   18  specified 
"management,"  and  9  indicated  "other."   The  respondents  were  selected  from  a 
mailing  list  of  users  of  the  Unigraphics  CAD  system  at  Eastman  Kodak  and  from 
personal  contacts  made  by  the  authors.  A  copy  of  the  survey  is  shown  in  Appendix 
A.  These  data  were  used  to  establish  the  imp>ortance  weighting  vector  for  the  study. 

Process/Part  Evaluation 

Parts  were  evaluated  along  the  three  dimensions  — performance,  time,  and  cost — 
for  each  of  four  processes.  Estimates  of  time  and  cost  were  made  by  experienced 
employees  at  KAD.   In  order  to  reduce  variability  due  to  communication  issues,  the 
majority  of  the  estimates  were  made  with  the  part  in  hand.  Table  2  lists  the 
assumptions  upon  which  the  estimates  were  based  and  the  components  of  the  time 
and  cost  estimates.  The  performance  of  each  part  was  calculated  using  the  attribute 
rating  system  described  in  Section  1,  Framework,  and  the  attributes  listed  in  Table  1. 

Table  2:  Components  of  Estimates  and  Assumptions. 


txDmponents  of  cost 

CAD 

STL 

CNC 

RM 

labor 

•  1  skilled  CAD 

•  1  skilled  CAD 

•  1  skilled  CNC 

•  1  skilled  mold- 

operator 

operator 
•  1  skilled  SLA 
Of)erator 

programmer 
•  1  skilled  CNC 
Of)erator 

maker 

machine 

•  CAD 

•SLA500 

•  5-axis  CNC  mill 

•  vacuum  pump 

workstation 

•  curing  oven 

•  CNC  lathe 

•  ventilation  hood 

materials 

•  Ciba-Geigy  resin 

•  fixture  material 

•  part  material 

•  mold  material 

•  resin  for  part 

•  gloves,  syringes, 
etc.  for  handling 
resins 
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components  of  time 


translation  of 
drawing  or 
defined  concept 
to  CAD  solid 
model 


•  generation  of 
support 
structures 

•  translation  and 
manipulation  of 
build  files 

•  build  time 

•  cure  time 

•  hand  finishing 


•  definition  of  tool 
path 

•  debugging  of 
program 

•  setup  time 

•  runtime 

•  hand  finishing 
and/or  post 
machining 
where  needed 


•  design  of  mold 

•  fabrication  of 
inserts,  supports, 
etc. 

•  fabrication  of 
mold 

•  mold  cure  time 

•  pKJuring  of  part 

•  part  cure  time 


A  number  of  factors  make  estimating  processing  time  and  cost  difficult.   First,  the 
reliability  and  consistency  of  the  estimator  had  to  be  established.   In  many 
organizations  estimating  is  still  more  art  than  science;  often  the  estimators  would 
produce  one  estimate  then  start  over  as  they  realized  a  more  elegant  solution  to  the 
fabrication  problem.   Second,  we  had  to  define  where  one  process  finished  and 
another  started,  and  we  had  to  define  the  "average"  target  level  of  performance  for 
each  process.  For  example,  a  stereolithography  part  destined  for  the  paper  path  of  a 
copier  required  more  hand  finishing  time  than  a  similar  stereolithography  part 
designed  to  hold  rolls  of  film.   Third,  the  skill  levels  of  the  operators  were  also  an 
issue.   Fourth,  we  often  found  variations  within  a  given  process;  a  given  part  can 
often  be  fabricated  by  a  given  process  a  number  of  different  ways.  Finally,  the 
capacity  utilization  of  a  process  determines  the  time  a  part  will  spend  waiting  for  the 
process.   For  the  time  estimates,  we  assumed  that  capacity  utilization  was  low 
enough  that  the  queueing  effects  were  negligible.   This  will  only  be  true  in 
organizations  that  allocate  enough  capacity  to  prototyping  that  jobs  move  through 
the  shop  without  contending  for  resources. 

Rather  than  spend  a  great  deal  of  time  calculating  an  exhaustive  estimate  with  a 
very  tight  margin  of  error,  the  estimators  were  instructed  to  maintain  consistency 
between  parts;  the  accuracy  of  one  part  relative  to  another  was  more  important  than 
the  estimate  for  any  given  part.   It  should  also  be  noted  that  the  estimators  were 
explicitly  told  that  they  would  not  be  held  to  their  estimates  (we  tried  to  decouple 
the  estimating  process  from  the  organizational  incentives  within  KAD).   Also,  the 
estimators  were  familiar  with  many  of  the  parts;  in  some  cases  they  had  fabricated 
the  parts  within  the  past  year. 

For  each  part,  one  of  us  (Wall)  rated  performance  in  each  attribute  category  on  a 
scale  of  0  to  1 .  The  basis  for  the  rating  was  the  answer  to  the  question,  "How  well 
does  the  prototype  part  approximate  the  production  part?"   In  some  cases,  the 
question  was,  "How  much  information  does  the  prototype  part  tell  about  the 
production  part?"   For  some  of  the  attributes  [and  for  the  CAD  process]  one  question 
makes  more  sense  than  the  other.   A  score  of  1  meant  that  the  prototype  part 
matched  the  production  part  for  that  attribute  (or  that  the  prototype  part  provided  as 
much  information  as  the  production  part).   For  example.  Tables  3a  &  3b  show  the 
estimates  for  and  evaluation  of  the  two  sample  parts  in  Figure  1. 
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Table  3a:  Example  of  Estimates 
(times  in  hoursrminutes) 


process         pnxess  component 


"bracket" 


"duct" 


CAD            time  to  create  solid  model 

4:00 

6:00 

STL             time  to  generate  support  structures 

i 

1:00 

0:15 

time  to  prepare  files  for  slicing  & 

0:30 

0:15 

building 

time  to  create  build  file 

0:15 

0:10 

time  to  build  part  on  SLA500 

3:45 

5:00 

time  to  cure  part  in  UV  oven 

2:00 

1:00 

time  to  hand  finish 

2.-00 

1:00 

CNC            time  to  program 

14:00 

7:00 

time  to  setup  and  debug 

12:00 

5:00 

time  to  run  1  part 

1:45 

1:30 

time  to  build  fixture 

0:00 

3:00 

number  of  sp>ecial  fixtures 

0 

1 

RM               time  to  build  sf)ecial  tooling 

8:00 

0:30 

time  to  setup  and  fabricate  mold 

12:00 

10:00 

time  to  pour  one  part 

1:00 

0:45 

time  to  cure  one  part 

12:00 

12:00 

time  to  cure  mold 

32:00 

32:00 

qty  of  part  material 

10  cubic  inches 

9  cubic  inches 

qty  of  mold  material 

150  cubic  inches 

132  cubic  inches 

number  of  pieces  to  mold 

2 

2 

Table  3b:  Exampl 

es  of  Evaluations 

"bracket" 

"ducr 

attribute 

CAD 

STL 

CNC 

RM 

CAD 

STL     CNC 

RM 

strength  /  stiffness 

0 

0.55 

0.9 

0.85 

0 

0.65      0.95 

1.0 

strength 

0 

0.5 

0.9 

0.9 

0 

0.6        0.9 

1.0 

stiffness 

0 

0.6 

0.9 

0.8 

0 

0.7        1.0 

1.0 

form  accuracy 

0.83 

0.85 

0.85 

0.98 

0.75 

0.85      0.83 

0.95 

dimensional  accuracy 

1.0 

0.8 

0.9 

0.9 

1.0 

0.8        0.9 

0.9 

geometry 

1.0 

0.9 

0.7 

1.0 

1.0 

0.9        0.8 

0.9 

surface  finish 

0.3 

0.8 

0.9 

1.0 

0.1 

0.8        0.8 

1.0 

feature  definition 

1.0 

0.9 

0.9 

1.0 

0.9 

0.9        0.8 

1.0 

appearance 

0.5 

0.8 

0.9 

1.0 

0.7 

0.8        0.8 

1.0 

overall  appearance 

0.5 

0.8 

0.9 

1.0 

0.7 

0.8        0.8 

1.0 

material  properties 

0.23 

0.52 

0.98 

1.0 

0.23 

0.37      0.98 

1.0 

hardness 

0 

0.9 

1.0 

1.0 

0 

1.0        1.0 

1.0 

weight 

0.9 

0.8 

0.9 

1.0 

0.9 

0.9        0.9 

1.0 

melting  temp 

0 

0 

1.0 

1.0 

0 

0         1.0 

1.0 

opacity 

0 

0.3 

1.0 

1.0 

0 

0.3        1.0 

1.0 

color 

0.5 

0 

1.0 

1.0 

0.5 

0         1.0 

1.0 

conductivity 

0 

0 

1.0 

1.0 

0 

0         1.0 

1.0 
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Once  these  evaluations  were  performed  for  all  104  parts,  the  part  performance 
ratings,  the  cost  estimates,  and  the  time  estimates  were  averaged  to  rank  the 
processes  in  terms  of  performance,  cost,  aind  time. 

3.  RESULTS 
Summary  of  Results 

The  results  of  the  study  are  summarized  in  Figures  2-10.  Figure  2  shows  the 
importance  of  each  aggregate  prototype  attribute  based  on  replies  to  the  Attribute 
Importance  Survey.    (The  individual  attributes  that  make  up  the  aggregate  attributes 
shown  in  Table  2  are  listed  in  Table  1.)  Note  that  the  importance  of  each  attribute 
increases  as  the  project  progresses  and  that  time  was  rated  most  important  at  every 
phase  of  product  development.   As  expected,  there  are  differences  in  the 
importances  indicated  by  the  respondents.   There  is  also  some  difference  in  the 
mean  responses  among  respondents  who  classified  themselves  as  "engineering"  or 
"design"  and  those  who  claimed  "management."   Depending  on  the  goals  of  the 
prototyping  evaluation  process,  a  weighting  vector  could  be  defined  for  a  single 
individual  or  for  a  particular  group  of  individuals.   For  presentation  purposes,  we 
created  a  weighting  vector  by  averaging  all  of  the  responses.  The  resulting  vector  is 

W^  =  [o.73  0.74  0.53  0.49]  for  the  attributes  strength /stiffness,  dimensional  accuracy, 

appearance,  and  material  properties  other  than  strength  and  stiffness. 

The  results  of  the  lead  time  estimates  from  the  Process /Part  Evaluation  are  shown 
in  Figure  3  for  a  lot  size  of  1  and  in  Figure  4  for  a  lot  size  of  25.  The  results  of  the 
cost  estimates  from  the  Part/Process  Evaluation  are  shown  in  Figures  5  for  a  lot  size 
of  1  and  in  Figure  6  for  a  lot  size  of  25.  Note  that  the  horizontal  axes  in  figures  3 
through  7  do  not  represent  continuous  variables,  rather  they  are  part  numbers  1 
through  104  evenly  spaced  along  the  horizontal  axis.   The  values  for  each  process 
are  connected  together  by  lines  to  make  the  data  legible  and  not  to  suggest  a 
continuous  function.   The  data  in  figures  3  through  7  were  all  sorted  in  order  of 
increasing  lead  time  for  the  CNC  process  and  a  lot  size  of  1.  Lead  time  to  fabricate  a 
single  part  for  each  of  the  processes  depended  loosely  upon  part  complexity  (where 
complexity  was  related  to  the  number  of  geometric  primitives  needed  to  define  the 
part)  and  so  the  parts  are  roughly  ordered  in  terms  of  increasing  complexity. 

The  scalar  part  performances  for  each  part  and  process  are  plotted  in  Figure  7.  The 
scalar  performance  of  each  part  was  calculated  using  a  performance  vector  (A),  based 
on  tiie  attributes  in  Table  1,  and  the  weighting  vector,  W^  =  [0.73  0.74  0.53  0.49], 
derived  from  the  Attribute  Importance  Survey.      The  data  were  sorted  as  in  Figures 
3-6. 

Averaging  of  the  part  performances  for  each  process  yields  the  average  process 
performance  results  shown  in  Figure  8.    For  each  process,  the  time  and  cost  scores 
are  normalized  values  relative  to  the  other  processes,  where  1  is  fastest /least 
expensive  and  0  is  slowest /most  expensive.  Time  and  cost  ratings  are  for  a  lot  size 
ofl. 
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Figvire  2:  Mean  and  Standard  Deviation  of  the  Importance 
of  Performance  Attributes,  Time,  &  Cost  at  Various  Phases  of  Development 
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Figure  3:  Lead  Time  for  Each  Part  (lot  size  of  1) 
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Figure  4:  Lead  Time  for  Each  Part  (lot  size  of  25) 
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Figure  5:  Cost  of  Each  Part  (lot  size  of  1) 
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Figure  6:  Cost  of  Each  Part  (lot  size  of  25) 
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Figure  7:  Aggregate  Performance  of  Parts 
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Figure  8:  Mean  and  Standard  Deviation 
of  Performance,  Time,  &  Cost  Ratings  for  Different  Processes 
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Finally,  Figures  9  and  10  illustrate  the  quantitative  relative  time-  and  cost- 
p>erformance  of  each  process  based  on  lot  sizes  of  1  and  25.  Note  that  becavise  the 
relative  ordering  of  the  cost  and  time  values  for  each  of  the  processes  is  roughly  the 
same  regardless  of  the  parts  (indicated  by  infrequent  crossing  of  the  process  lines  in 
figures  3  through  6),  the  large  standard  deviations  in  figures  9  and  10  do  not  imply 
an  inability  to  predict  relative  overall  process  performance.  Rather,  a  part  costing 
substantially  less  than  the  mean  to  produce  by  STL  v^ll  likely  also  cost  substantially 
less  than  the  mean  to  produce  by  CNC. 

Interpretation  of  Results 

A  decision  among  processes  for  prototyping  a  particular  part  can  be  made  based  on 
the  data  in  Figures  8-10.  The  first  step  is  to  determine,  using  Figure  8,  w^hich 
processes  w^ill  provide  the  information  required  of  the  prototype.   Then  Figures  9  & 
10  are  used  to  determine  which  of  these  processes  is  best  in  terms  of  time  or  cost  for 
a  given  lot  size.   For  example,  the  bracket  shown  in  Figure  1  must  be  made  of  a 
conductive  material.   Based  on  this  fact  and  the  process  ratings  in  Figure  8,  the 
alternative  processes  are  RM(CNC),  RM(STL),  and  CNC.  If  we  assume  that  25  parts 
are  required  and  that  time  is  the  driving  factor  in  the  process  decision.  Figure  9 
shows  that  CNC  is  the  preferred  process.  If  we  extend  the  example  and  assume 
instead  that  cost  were  the  driving  factor.  Figure  10  shows  that  rubber  molding  from 
a  stereolithography  master  is  preferred. 

Table  4  summarizes  the  process  decisions  for  a  small  set  of  decision  drivers.   The 
table  uses  time,  cost,  the  four  aggregate  attributes,  and  lot  size  as  the  factors  that 
determine  which  process  to  use.  Each  entry  in  the  table  was  made  by  (1)  narrowing 
the  set  of  possible  processes  to  those  that  will  perform  well  with  respect  to  a  critical 
attribute  (using  Figure  8),  and  (2)  determining  the  best  time  or  cost  performance  for 
the  appropriate  lot  size  (using  Figures  9  &  10).   The  parenthesized  entries  in  the 
"form  accuracy"  row  indicate  the  preferred  process  if  a  physical  part  is  required.  In 
some  cases  either  STL  or  CNC  would  be  the  "preferred"  process;  whereas  the 
average  values  indicated  one  process,  the  standard  deviations  were  large  enough  to 
suggest  that  the  other  might  be  a  viable  alternative.   In  these  cases  alternatives  are 
provided  in  parentheses. 
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Figitre  9:  Mean  and  Standard  Deviation  of 
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Figure  10:  Mean  and  Standard  Deviation  of 
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Table  4:  Preferred  Prototyping  Processes  Based  on  Results. 


critical  attribute 

Cases  where  time 
Qty:  1 

is  the  driver 
Qty:  25 

Cases  where  cost 
Qty:  1 

is  the  driver 
Qty:  25 

strength  /  sti  f  f  ness 

CNC 

CNC 

RMfromSlL 
(CNC) 

RM  from  STL 
(CNC) 

form  accuracy 

CAD 
(STL) 

CAD 
(CNC) 

CAD 
(STL) 

CAD 
(RM  from  STL) 

appearance 

STL 

CNC 

STL 

RM  from  STL 
(STL  or  CNC) 

material 
properties 

CNC 

CNC 

RMfromSlL 
(CNC) 

RM  from  STL 

(CNC) 

Table  4  lists  the  guidelines  for  a  particular  class  of  parts,  processes,  attributes,  and  lot 
sizes,  but  the  procedure  described  above  can  be  applied  more  generally  to  different 
classes  of  parts,  attribute  sets,  weighting  vectors,  and  lot  sizes.  The  procedure  for 
evaluating  processes  is  thus: 

(1)  Determine  v^^hich  of  the  performance  attributes  are  most  critical. 

(2)  Determine  vy^hich  processes  produce  parts  which  have  adequate 
performance  for  these  attributes. 

(3)  Choose  between  processes  based  on  time,  cost,  or  both  (depending  on 
the  relative  importance  of  time  and  cost  for  the  part  procurement). 

4.  CONCLUSIONS  &  DIRECTIONS  FOR  FUTURE  WORK 

We  have  evaluated  a  set  of  prototyping  technologies  for  a  focused  domain  of 
applications.  In  doing  so  we  have  demonstrated  a  methodology  for  assessing  and 
representing  performance  needs  for  a  prototype  part  and  for  evaluating  different 
processes  with  respect  to  these  needs.  We  have  been  able  to  prescribe  the  most 
promising  prototyping  processes  for  a  specific  situation,  given  the  performance 
characteristics  that  are  most  important  for  the  part  and  given  either  a  time-  or  cost- 
driven  decision  process.  In  the  course  of  this  study,  we  have  also  developed  insights 
into  some  of  the  broader  issues  in  prototyping.   In  this  final  section  we  present  some 
of  these  insights  and  discuss  some  areas  for  future  work  in  supporting  design 
decision  making  and  in  improving  the  prototyping  processes  themselves. 

Insights  from  the  Prototyping  Study 

Time  is  almost  always  very  important  to  the  customer  of  a  prototyping  process. 
This  result  resonates  with  recent  emphasis  on  shortened  product  development 
cycles.   It  is  also  consistent  v^dth  findings  in  [Ulrich91]  indicating  that  in  some 
product  contexts,  the  economic  impact  of  lead  time  dominates  materials  cost  and 
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labor  cost  issues  in  design  decision  making.  The  criticality  of  time  suggests  that 
firms  should  maintain  enough  prototyping  capacity  that  queueing  delays  do  not 
influence  the  lead  time  of  the  product.  Justifying  the  associated  capital  outlay  is 
difficult  without  some  explicit  estimate  of  the  value  of  prototyping  lead  time  to  the 
organization. 

Conventional  CNC  machining  is  among  the  best  prototyping  alternatives.   For 
plastic  parts,  process  performance  depends  on  the  details  of  the  application. 
Nevertheless,  it  is  interesting  that  the  best  choice  of  prototyping  technology  (Table  4) 
is  dominated  by  CNC  machining  for  the  time-driven  case.   We  have  observed  that 
many  designers  are  surprised  by  this  result  because  the  trade  literature  has  given  a 
lot  of  attention  to  so-called  rapid  prototyping  processes.    Although  we  see 
tremendous  opportunities  for  rapid  prototyping  technologies  like  stereolithography, 
machining  is  often  the  best  available  prototyping  technology.   In  our  subsequent 
discussion  of  process  development  we  will  articulate  the  process  developments  that 
may  change  the  relative  strengths  of  these  processes. 

Rapid  is  a  relative  term  for  prototyping.   The  promise  of  instant  models  from  CAD 
data  is  still  largely  unfulfilled.  The  average  lead  time  to  acquire  a  physical  prototype 
from  stereolithography  was  9.3  hours  for  our  part  sample,  assuming  that  there  was 
no  queue  for  the  stereolithography  machine.   However,  in  general  STL  parts  can  not 
be  used  in  functional  tests  because  of  the  relatively  poor  material  properties  of  the 
stereolithography  resins.  The  average  time  to  acquire  a  part  with  good  stiength  and 
stiffness  properties  was  24.2  hours  (CNC  machining).    TTiis  can  be  as  long  as  three 
calendar  days  for  a  one-shift  prototype  shop. 

Processes  tend  to  perform  better  on  simple  parts  than  complex  parts.  This  result  is 
intuitive  but  because  we  have  not  displayed  the  geometry  of  each  of  the  104  parts,  it 
is  not  clear  from  the  presentation  of  our  data.   In  general,  the  differences  between 
processes  were  greater  than  the  differences  in  time,  cost,  and  performance  between 
parts.   (In  fact,  this  is  what  allows  us  to  make  the  part-independent 
recommendations  in  table  4.)     Nevertheless,  simpler  parts  (measured  by  the 
number  of  geometric  primitives  in  the  CAD  file)  in  general  obtained  higher 
performance  ratings.   One  implication  of  this  observation  is  that  part  designers 
might  consider  simplifying  or  tailoring  part  geometry  in  anticipation  of  prototyping 
requirements.  A  related  idea  was  suggested  for  analysis  tools  in  [Suri89]  and  seems 
equally  appropriate  for  prototyping. 

There  is  still  no  substitute  for  a  skilled  operator.  Many  views  of  "desktop 
manufactUi-ing"  neglect  the  knowledge  and  skills  of  machine  operators  by  trying  to 
make  the  designer  omniscient.   We  found  that  many  ideas  are  born  as  designers 
interact  with  fabricators. 

Supporting  Design  Decision  Making 

A  complex  electromechanical  product  may  contain  thousands  of  parts  with  diverse 
intended  functionality  and  diverse  intended  final  production  processes.    When 
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integrated  into  the  design  process,  a  rough-cut  prototyping  process  evaluation 
system  could  be  a  powerful  tool  for  the  designer,  engineer,  and  manager.   A 
computerized  implementation  may  be  imagined  as  follows.   As  the  project  is 
irutiated,  attributes  and  importances  are  defined  and  agreed  upon  for  different 
classes  of  parts.  Decisions  about  the  number  of  prototypes  that  will  be  required  at 
various  stages  of  the  process  are  established.  This  information  can  be  used  to 
inform  the  designer  what  the  recommended  prototyping  process  is  and  what  the 
associated  time,  cost,  and  performance  measures  for  the  current  part  or  assembly  are. 
The  designer  could  then  add  this  information  to  the  information  available  for  other 
evaluation  criteria  in  order  to  make  better  detailed  part  design  decisions.   As  the 
needs  of  the  project  evolve,  the  project  team  might  change  the  desired  lot  sizes  or 
call  for  re-evaluation  of  attribute  importances.   Such  a  tool  would  fit  naturally  into  a 
group  design  environment  utilizing  comprehensive,  interactive  design  software. 

Improving  Prototyping  Processes 

Figures  11  and  12  and  Table  5  summarize  the  process  cost  and  time  components  for 
CNC,  STL,  and  RM.   Along  with  the  average  process  p>erformance  results  shown  in 
Figure  8,  these  results  help  to  focus  prototyping  process  development.      Note  that  in 
this  case  the  STL  and  CNC  figures  do  not  include  the  time  or  cost  to  create  a  CAD 
model  and  the  RM  figures  do  not  include  the  time  or  cost  to  fabricate  a  master. 
Note  that  the  lead  times  for  25  parts  could  be  reduced  substantially  in  some  cases  by 
parallel  processing.   For  example,  if  the  parts  are  small  enough,  more  than  one  can 
be  built  simultaneously  on  the  SLA500  platform.   Or,  if  more  than  one  master  is 
available,  multiple  parts  can  be  molded  simultaneously. 

Table  5:  Average  Lead  Times  (hours) 

STL CNC RM 

1  part  9.3  24.2  59.2 

25  parts  130.9  88.8  370.9 

Each  of  the  processes  has  weaknesses  and  could  benefit  immensely  from  research 
and  development.    The  figures  illuminate  important  areas  for  process  development: 

•  Nearly  50%  of  the  time  to  fabricate  one  CNC  part  is  programming 
time.  If  that  is  reduced  (by  automated  generation  of  cutter  paths,  for 
example)  then  CNC  becomes  even  more  useful  as  a  prototyping  tool. 

•  STL  performs  very  well  on  average  except  where  material  properties 
are  critical;  the  available  resins  are  too  limited.    An  improvement  in 
resins  would  immediately  make  STL  the  dominant  prototyping 
technology  for  plastic  parts. 

•  CAD  is  very  fast  and  has  high  geometrical  accuracy.  But,  it  falls  short 
in  evaluating  strength,  stiffness,  and  other  material  properties.    We 
did  not  assess  finite  element  analysis  methods  of  prototyping  which 
address  some  of  these  weaknesses.  However,  the  state-of-the-art  in 
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simulation  is  still  an  inadequate  substitute  for  a  physical  model. 
Advances  in  non-linear,  dynamic  simulation  of  multiple  parts  would 
enable  CAD  to  emerge  as  an  even  more  important  prototyping 
technology. 

•  Rubber  molding  is  an  attractive  process  because  it  can  be  used  for 
small  scale  production.   But  the  process  is  slow.  There  is  a  dear  need 
for  either  fast-curing  polymers  or  for  technologies  that  allow 
thermoplastics  to  be  molded  from  quickly-procured  tooling. 
Developments  in  spray  metal  tooling  may  eventually  enable  the 
fabrication  of  batches  of  100  plastic  parts  relatively  quickly  [Weiss90]. 

We  hope  that  the  specific  results  from  our  study  soon  will  be  obsolete  because  of 
rapid  advances  in  process  development.   We  believe,  however,  that  the 
methodology  we  have  developed  can  be  used  to  make  intelligent  and  rational 
prototyping  decisions  as  processes  change  and  in  differing  product  development 
contexts.   The  methodology  supports  the  tactical  decisions  surrounding  part  design 
and  choice  of  prototyping  technologies  as  well  as  the  strategic  decisions  surrounding 
investment  in  process  development. 
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Figure  11:  Components  of  Cost 


Figiuel2:  Process  Components 
•5  a  Fraction  of  Time  (lot  size  of  1) 
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6.  APPENDIX  A  -  THE  ATTRIBUTE  IMPORTANCE  SURVEY 

Name:  ^^^1^ '^"V.Tn"^ 

Matthew  Wall 

I^ate:  2-4-EP  mail  code:  35326 

LoB: 

Primary  Function  (choose  one):  Primary  Discipline  (choose  one): 

O    engineering  O    mechanical 

O    design  O    electrical 

O    management  O    optical 

O    other O    other 


At  what  stage(s)  of  the  product  development  cycle  are  you  involved  with  prototype 
design,  fabrication,  or  assembly? 

O  concept  development 
O  product  engineering 
O    pre-production 

Given  that  prototypes  are  built  for  concept        engineering       pre-prod 

different  purposes,  how  important 

(on  average)  are  the  following  c 

attributes  in  your  prototype  parts  t 

during  the  phases  of  product  g 

development  listed  to  the  right?  '| 


c 

s 

c 

C 
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c 

c 
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■§ 

s. 

■2 
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6 

§. 

E 

g. 

E 

if 

E 

£■ 
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er 

$ 

c 

9 

§! 

c 

u 

stiffness  or  compliance  OOO  OOO  OOO 

dimensional  accuracy  OOO  OOO  OOO 

geometry  OOO  OOO  OOO 

surface  finish  OOO  OOO  OOO 

hardness  OOO  OOO  OOO 


manufacturing  tolerance  variability  OOO  OOO  OOO 

weight  OOO  OOO  OOO 

melting  temperature  OOO  OOO  OOO 

opacity  or  transparency  OOO  OOO  OOO 

color  OOO  OOO  OOO 


conductivity  or  resistivity  OOO  OOO  OOO 

method  of  integration  into  system  OOO  OOO  OOO 

strength  OOO  OOO  OOO 

feature  definition  OOO  OOO  OOO 

overall  appearance  OOO  OOO  OOO 


functionality  OOO  OOO  OOO 

cost  OOO  OOO  OOO 

turnaround  time  OOO  OOO  OOO 

manufacturing  process  OOO  OOO  OOO 

assembly  procedure  OOO  OOO  OOO 

Please  list  some  critical-to-function  features  typically  found  in  your  prototype  parts. 
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